Papers with Theoretical analysis
Thesis Proposal: Efficient KV Cache Reuse for Multi-Document Retrieval-Augmented Generation (2026.eacl-srw)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) systems face efficiency bottlenecks in prefill due to attention mechanism, and traditional KV cache only accelerates decoding. |
| Approach: | They propose a multi-document KV cache reuse framework for multi-doc RAG workloads . they propose to resolve position and context misalignment while eliminating document-specific quadratic complexity in prefill. |
| Outcome: | The proposed framework solves position and context misalignment issues while eliminating document-specific quadratic complexity in prefill. |
KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Empirical evaluation shows that our approach yields superior performance in both standard task metrics and large language model (LLM)-based evaluation. |
| Approach: | They propose a K-step return estimation method for reinforcement learning (RL)-based knowledge distillation in text generation tasks using the Bellman Optimality Equation. |
| Outcome: | The proposed method performs better on standard task metrics and large language model evaluations on three text generation tasks. |
Dice Loss for Data-imbalanced NLP Tasks (2020.acl-main)
Copied to clipboard
| Challenge: | Using dice loss, we find that data imbalance is a common issue in many NLP tasks . data imbalance affects the performance of many tasks, such as tagging and machine reading comprehension . |
| Approach: | They propose to use dice loss to replace the standard cross-entropy objective for data-imbalanced NLP tasks. |
| Outcome: | The proposed training objective achieves significant performance boost on a wide range of data imbalanced tasks. |
LAGCL4Rec: When LLMs Activate Interactions Potential in Graph Contrastive Learning for Recommendation (2025.findings-emnlp)
Copied to clipboard
Leqi Zheng, Chaokun Wang, Canzhi Chen, Jiajun Zhang, Cheng Wu, Zixin Song, Shannan Yan, Ziyang Liu, Hongwei Li
| Challenge: | Traditional contrastive learning methods treat negative feedback as equally hard or easy, ignoring informative semantic difficulty during training. |
| Approach: | They propose a framework leveraging Large Language Models to Activate interactions in Graph Contrastive Learning for Recommendation. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on multiple benchmarks. |
TeRA: Vector-based Random Tensor Network for High-Rank Adaptation of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) methods have significantly reduced the number of trainable parameters needed in fine-tuning large language models. |
| Approach: | They propose a vector-based random Tensor network for high-Rank Adaptation method that achieves high-rank weight updates while retaining parameter efficiency. |
| Outcome: | The proposed method outperforms existing PEFT methods while keeping low-rank parameters. |
WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods focus on distinguishing fully watermarked text from non-watermarked text, overlooking real-world scenarios where LLMs generate only brief segments within longer documents. |
| Approach: | They propose a method to detect watermarked segments in large documents using an anomaly extraction method and a local traversal. |
| Outcome: | The proposed method achieves a superior balance between detection accuracy and computational efficiency. |
RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on retrieval-augmented language models has two main problems: determining what information to retrieve and effectively combining retrieved information during generation. |
| Approach: | They propose a retrieval-augmented language model that captures current and future information from source and target text into a latent space. |
| Outcome: | The proposed model is more efficient than explicit raw text, but limited by context length and noise. |
FaithLM: Towards Faithful Explanations for Large Language Models (2026.eacl-long)
Copied to clipboard
Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Ruixiang Tang, Shaochen Zhong, Fan Yang, Andrew Wen, Mengnan Du, Xuanting Cai, Vladimir Braverman, Xia Hu
| Challenge: | Large language models (LLMs) produce natural language explanations, but they lack faithfulness and do not reflect the evidence the model uses to decide. |
| Approach: | They propose a model-agnostic framework that evaluates and improves the faithfulness of LLM explanations without token masking or task-specific heuristics. |
| Outcome: | The proposed framework improves faithfulness of large language models without masking or heuristics. |
Stable Language Guidance for Vision–Language–Action Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing vision-Language-Action models are notoriously brittle to linguistic perturbations. |
| Approach: | They propose a probabilistic framework that disentangles physical affordance from semantic execution. |
| Outcome: | The proposed framework disentangles physical affordance from semantic execution. |
Coming to Your Senses: on Controls and Evaluation Sets in Polysemy Research (D18-1)
Copied to clipboard
| Challenge: | a prior art claim that sense-specific vectors provide an advantage over normal vectors is unfounded in two ways. |
| Approach: | They claim that sense-specific vectors provide an advantage over normal vectors due to the polysemy that they presumably represent. |
| Outcome: | The proposed results show that ground-truth polysemy degrades performance in word similarity tasks and that random assignment of words to senses improves performance. |
Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing unified optimization strategies overlook the statistical conflict between these distinct gradient signals. |
| Approach: | They propose a framework to reduce bias-variance trade-offs in Large Language Models . they propose DYPO, which leverages intrinsic group dynamics to significantly reduce RL gradient variance . |
| Outcome: | The proposed framework outperforms traditional pipelines on reasoning benchmarks and out-of-distribution tasks. |
Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) can bridge the gap between large language models and downstream tasks, but is vulnerable to malicious attacks. |
| Approach: | They propose a weak-to-strong unlearning algorithm based on feature alignment knowledge distillation to defend against backdoor attacks . they first train a small-scale language model through full-parameter fine-tuning to serve as the clean teacher model and then guide the large-scale poisoned student model in unlearning the backdoor. |
| Outcome: | The proposed method can unlearn backdoor features without compromising model performance. |
ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | NVFP4 supports fine-grained block isolation, 4-bit quantization errors and mixed-precision approaches . ARCQuant boosts NVFO4 performance via Augmented Residual Channels . |
| Approach: | They propose a framework that boosts NVFP4 performance via Augmented Residual Channels. |
| Outcome: | ARCQuant boosts NVFP4 performance via Augmented Residual Channels . the proposed framework achieves state-of-the-art accuracy comparable to full-precision baselines compared to FP16 . |
Visual Attention Reasoning via Hierarchical Search and Self-Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) often hallucinate due to fragile, linear reasoning and weak visual grounding. |
| Approach: | They propose a framework that reformulates reasoning as a hierarchical search with self-verification and replaces linear Chain-of-Thought with a tree-search policy capable of backtracking to correct logical errors. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on hallucination and safety benchmarks. |
Leveraging 3D Gaussian for Temporal Knowledge Graph Embedding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Representation learning in knowledge graphs (KGs) has focused on static data, yet many real-world knowledge graph are inherently dynamic. |
| Approach: | They propose a temporal embedding method inspired by 3D Gaussian Splatting where entities, relations, and timestamps are modeled as 3D gaussian distributions with learnable structured covariance. |
| Outcome: | The proposed method outperforms state-of-the-art methods on three benchmark TKG datasets. |
R-AT: Regularized Adversarial Training for Natural Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, adversarial training is a popular and powerful regularization method in the natural language domain. |
| Approach: | They propose to regularize adversarial training via dropout by perturbing word embeddings . they find that R-AT can improve many models by reducing adversariality . |
| Outcome: | The proposed method can reduce the inconsistency between training and testing of models with dropout. |
SMR: State Memory Replay for Long Sequence Modeling (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing state space models (SSMs) address non-uniform sampling, but their recursive structures impede efficient SSM computation via convolution. |
| Approach: | They propose a plug-and-play mechanism to solve the Non-Stable State problem by adjusting input sequences with early memories. |
| Outcome: | The proposed method overcomes the non-uniform sample processing problem . it can achieve Sampling Step Adaptation (SSA) by adjusting input sequences with early memories. |
Instruction Position Matters in Sequence Generation with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can perform conditional sequence generation tasks, such as translation or summarization, through instruction fine-tuning. |
| Approach: | They propose to shift the position of task instructions after the input sentences to enhance the model's instruction-following capability. |
| Outcome: | The proposed method outperforms traditional settings across various model scales (1B / 7B & 13B) and different sequence generation tasks (translation and summarization) without any additional data or annotation costs. |
Learning Continuous Temporal Dynamics on Symplectic Manifolds for Temporal Knowledge Graph Embedding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for temporal knowledge graph embedding lack explicit structural constraints for continuous-time dynamics. |
| Approach: | They propose a Temporal Knowledge Graph Embedding framework that embeds temporal dynamics into a symplectic phase space. |
| Outcome: | The proposed framework achieves competitive performance with lower embedding dimensions. |
GMFL: Efficient Global Masking for Federated LLM Fine-tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) has emerged as a prominent solution to mitigate the communication and computation costs in federated fine-tuning of Large Language Models (LLMs). |
| Approach: | They propose a plug-and-play layer freezing mechanism to integrate with existing federated fine-tuning frameworks. |
| Outcome: | The proposed solution reduces communication overhead and lowers computational costs while preserving the performance of the underlying federated fine-tuning methods. |
Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems (2025.findings-emnlp)
Copied to clipboard
Minghang Zhu, Zhengliang Shi, Zhiwei Xu, Shiguang Wu, Lingjie Wang, Pengjie Ren, Zhaochun Ren, Zhumin Chen
| Challenge: | Existing methods for fine-tuning agents are often inadequate . a multi-agent system can solve complex tasks by dividing responsibilities among specialized agents . |
| Approach: | a new framework is proposed to improve agents collaboration through iterative alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on held-in and held-out tasks. |
Localized Low-Rank Adaptation within Clustered Parameter Subspaces (2026.acl-long)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) for large language models has been successful in various domains. |
| Approach: | They propose to perform low-rank updates within clustered parameter subspaces . they group rows/columns of update matrix into locally coherent, uncorrelated subspace blocks . |
| Outcome: | Empirical results show that low-rank Adaptation (LoRA) is better than global adaptations in various domains. |
Beyond Compromise: Pareto-Lenient Consensus for Efficient Multi-Preference LLM Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent approaches to align LLMs with diverse human values are based on static linear scalarization or rigid gradient projection . however, these approaches often sacrifice potential global Pareto improvements to avoid transient local trade-offs. |
| Approach: | They propose a game-theoretic framework that reimagines alignment as a dynamic negotiation process. |
| Outcome: | The proposed framework breaks the deadlock between static linear scalarization and rigid gradient projection . it allows the model to escape local degradation and explore the distal Pareto-optimal frontier . |